WINGS: A Parallel Indexer for Web Contents
نویسندگان
چکیده
In this paper we discuss the design of a parallel indexer for Web documents. By exploiting both data and pipeline parallelism, our prototype indexer efficiently builds a partitioned inverted compressed index, a suitable data structure commonly utilized by modern Web Search Engines. We discuss implementation issues and report the results of preliminary tests conducted on a SMP PCs.
منابع مشابه
Parallel Clustering System Using the Methodologies of Evolutionary Computations
Several versions of the parallel clustering system were studied to improve performance of its initial implementation. The current versions were restricted to 1024 Web pages which, in turn, were used to create adaptive probe sets that were distributed to each indexer node. The probe sets were used to compute fitness measures associated with each indexer node used to create sub-species for the pu...
متن کاملDynamic Load Balancing Model: Preliminary Assessment of a Biological Model for a Pseudo-search Engine
Emulation of the current World Wide Web (WWW) search engines using methodologies derived from Genetic Programming (GP) and Knowledge Discovery in Databases (KDD) were used for the PseudoSearch Engine's initial parallel implementation of an indexer simulator. The indexer was implemented to follow some of the characteristics currently implemented by AltaVista and Inktomi search engines who index ...
متن کاملModels and Algorithms for Parallel Text Retrieval
MODELS AND ALGORITHMS FOR PARALLEL TEXT RETRIEVAL Berkant Barla Cambazoğlu Ph.D. in Computer Engineering Supervisor: Prof. Dr. Cevdet Aykanat January, 2006 In the last decade, search engines became an integral part of our lives. The current state-of-the-art in search engine technology relies on parallel text retrieval. Basically, a parallel text retrieval system is composed of three components:...
متن کاملA Combination Indexing for Image Social Bookmarking System to Improve Search Results
Web 3.0 and social bookmarking have altered the traditional roles of the indexer and user. Recently, web, allows users to create, organize, and search for images and other information sources through social tagging and other method activities. One of the image social bookmarking is such as Flickr. This research examines to increase the efficiency of image search result by creating indexes. The ...
متن کاملDesign of Building Automatic Global Concept Indexer for Ontology Alignment
Users of current World Wide Web (WWW) themselves have to involve in refining their search queries in order to find the exact answers because current WWW is web of documents representing only text, audio, video, images and metadata information (unstructured data) not conceptual information. Computers are used to present those documents only and not for retrieving the desired results which ultima...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2004